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value of online feedback systems in the context of widening participation and TEF. 
Significantly expanding the discussion beyond plagiarism, taking a genre-based 
approach, and positioning both academic writing and Turnitin/ feedback within the 


Seba context of academic literacies, this paper intervenes with current debates. The case 
on i study draws on qualitative data recorded from students, tutors, and the Turnitin 
Learning development : ie f 
software system. By doing so, insights are generated into best software practice that 
Software h Paina tase eel for. HEI ‘athe 4h he. aident 
Writing ave profound implications for s, most especially those with widening 


participation agendas. Based on these data, the study provides a series of practical 
software development recommendations to help raise standards amongst student 


writing. 


1. Introduction 


Within the Higher Education learning environment, the online formative and summative feedback 
process in platforms such as Turnitin is most effective when both students and staff are actively 
involved (Taras, 2003). However, students may fail to understand or interpret the feedback provided 
(Duncan et.al, 2007; Taras, 2003) and, practitioners frequently foreground deficits at the expense of 
developmental approaches (Hattie & Timperley, 2007). Semi-structured focus group interviews 
have demonstrated a bifurcation between student and staff perceptions of the use-value of Turnitin 
as a platform for learning, with only 27% of responses with positive connotations being initiated by 
staff. Extending beyond pedagogical debates into a software specification approach, this study 
provides new insightful recommendations for enhancing the use of software such as Turnitin as a 
potentially pedagogically impactful platform for all its users. 

The provision of high-quality assessment feedback to promote improvements in written 
assessments is closely aligned with the current direction of UK Higher Education policy; this is 
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especially pertinent for Higher Education Institutions (HEIs) with widening participation agendas 
(Department for Business Innovation and Skills [BIS], 2016a). As a particular measure of this, in May 
2016, the Department for Business Innovation and Skills (BIS) published its Technical Consultation 
document for UK universities (BIS, 2016a), part of its newly-devised Teaching Excellence Framework 
(TEF). The Technical Consultation document - being consultative in tone - offers a series of possible 
markers for success, rather than concrete definitions. However, it is particularly clear in its wish to 
support ‘the development, progression and attainment of students from disadvantaged 
backgrounds is a mark of effectiveness and therefore a key focus of the TEF’ (BIS, 2016a). To be sure, 
the TEF aims to measure the effectiveness of educational outcomes for students in Higher Education 
and address the needs of employers in a ‘knowledge economy’ (BIS, 2016b). The framework 
specifically seeks to promote improvements in student learning outcomes (ibid., p. 12). In keeping 
with this rationale, recent studies have explored how far structured feedback mechanisms, such as 
through formative assessment, can enable positive student assessment outcomes in the context of 
progressive approaches to assessment (Yorke, 2003; Butler & Roediger, 2008). Exploring the systems 
to provide structured feedback meaningful feedback, then, responds to both emerging UK 
educational policy and current scholarly debates. 

More specifically, the TEF Technical Document identifies a link between structured student 
feedback, and subsequent progression and attainment (BIS, 2016a). Accordingly, Wingate, Andon 
and Cago (2011) - in their study of the practice of embedding the teaching of academic writing 
conventions into course curricula - note that structured feedback leads to increased assessment 
grades and therefore a greater chance of student progression (Wingate, Andon and Cago, 2011, p. 
73). However, they find that - while most students approve of the formative feedback process - it 
can be impractical to deliver detailed face-to-face feedback from online submissions for larger 
cohorts. They decide that, since the formative feedback process for academic writing interventions 
is so labour intensive, ‘additional resources’ may be needed, including standardisation of feedback 
practice (Wingate, Andon and Cago, 2011, p. 73). One solution for the problem identified by Wingate 
may lie in providing at least partially-automated online feedback on features of academic writing; 
such a feature, in the context of TEF, could allow HEIs to demonstrate their pursuit of student 
progression through structured feedback. 

In keeping with this, Yang’s (2011) study focuses on a number of second-language science 
students to explore whether online peer assessment may be as an additional resource for feedback 
on writing. The system described by Yang involves an active visualisation (and discussion) of how 
peers solve problems through writing, articulated in a way that makes the descriptions of writing 
processes explicit and clear (Yang, 2011, p. 629). However, Yang cites a number of studies (Cho & 
Schunn, 2007; Liou & Peng, 2009; Lundstrom & Baker, 2009; Storch, 2005) that highlight how peer 
assessment of academic writing may result in discussion of lexical or syntactical features, which 
Lavelle and Zuercher (2001) dismiss as “surface concerns’ (p. 376). The study acknowledges further 
shortcomings arising from the expectation that students simply write like their peers (Braine, 1997; 
Paulus, 1999), as opposed - presumably - to adopting the styles and structures of writers in their 
wider discourse community. Yang’s solution to this is to encourage students to make larger-scale 
structural changes to peers’ writing as part of the coaching process (Yang, 2011, p. 268). This focus 
on structural features echoes the findings of Amos and McGowan (2012), who advocate analysis of 
the various sections of a text as a means to explore more general concepts of academic writing genres 
(Amos & McGowan, 2001, p. 2). However, in the only text extract Yang supplies, 7/8 of the peer 
corrections are surface-level changes, with only one of the example changes being structural, 
although a later description of student interactions mentions local and global changes equally (Yang, 
2011, p. 697). Overall, though, Yang’s study places slightly more emphasis on local/surface-level 
changes (38 mentions) than global changes (35 mentions). Clearly, whilst online peer feedback for 
writing promotes active dialogue on writing decisions, student-teacher interactions can enable a 
more constructive learning process based on a more sophisticated dialogue that simultaneously 
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addresses a text’s language, purpose, and structure (Amos & McGowan, 2012; Lavelle & Zuercher, 
2001; Nattinger & DeCarrico, 1992; Swales, 1990). 

Online systems like Turnitin’s Feedback Studio have been identified as means to improve the 
quality of work that students submit. Related to this, recent studies concerned with Feedback Studio 
have foregrounded its capacity as an educative tool that improves students’ understanding of 
academic misconduct. Academic writing, and the ability of students to appreciate feedback as a 
significant component of learning is often hidden behind the technological platform of Feedback 
Studio. In many cases, Feedback Studio is conceived as software used to detect dishonesty and frame 
students for inappropriate citation, or misuse of referencing. What are the student stories that 
manifest from the technology we use? How do we engage with students and help them see the value 
of technology in aiding their academic and professional development? This paper explores the use 
of Turnitin software as a means to enhance student learning through feedback and feedforward 
practices. 


1.1. Feedback Culture and Directed Learning 


The current corpus of research on feedback acknowledges its role in directing learning, as well as 
being an effective tool for learning (Hattie & Timperely, 2007; Shute, 2008). The seminal review by 
Black and William (1998), considered 250 studies on formative feedback and concluded that good 
and extensive feedback leads to increased student engagement and higher quality learning. 

Hattie and Timperley (2007) define feedback as information provided by an agent regarding 
aspects of one’s performance or understanding. Shute (2008) posits that the intention of providing 
students with feedback is to close the gap between the standard achieved and the standard desired. 
This is compounded by students who feel that feedback is meaningless and provides little direction 
for future improvements. How often do students read the feedback or actively seek out 
opportunities for feedback? Does online feedback change the dynamics of how students see and 
understand feedback comments? Duncan et.al, (2007) offers a possible analysis of how the student- 
assessor feedback nexus operates. According to Duncan et.al, (2007), part of the problem is that 
academics and students see feedback in isolation to other components of the teaching and learning 
process, and consider feedback to be primarily a teacher-owned process. This is corroborated by 
Taras (2003) who explains that the feedback process is most effective when all the protagonists are 
actively involved. A further problem identified by both Duncan et.al, (2007) and Taras (2003), is that 
students often fail to understand or interpret the feedback provided. This could, potentially, be due 
to the language used by academic staff in providing feedback comments. Hattie & Timperley (2007) 
find that many academics, when providing feedback, tend to focus on the correctional or deficit 
elements within the student work at the expense of providing instructional and developmental 
guidance. Studies on feedback impact on student learning achievement indicate that feedback has 
the potential to significantly enhance learning through feedback quality and not necessarily 
quantity. Hattie and Timperely (2007) also suggest that learning gains are heightened when feedback 
is directed towards developmental improvements as opposed to feedback that focusses on praise 
and rewards. Likewise, in their study of the psychology of learning development, Brown, Roediger 
and McDaniel, (2014) problematise approaches to learning that rely solely on students’ perspectives, 
and emphasise a potential gap between perceptions of learning needs versus meaningful acquisition 
of learning gains, which can be and uncomfortable process for the learner. Accordingly, feedback is 
most effective when it addresses realistic and achievable goals and does not threaten student self- 
esteem; feedback systems such as Turnitin may be able to play a part in this. 


1.2. Feedback for Learning and Transition 


New students, or students new to higher education learning environments, often face the challenges 
of transition. The literature suggests that there are multiple gaps in prior student learning 
experiences and those of higher education expectations (Rolfe, 2011). What students believe they 
understand in terms of academic skills are often poles apart from academic staff expectations 
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(Taylor, 2008). Rolfe (2011) suggests that the difference in the experience - expectation continuum, 
could be due to cultural shifts and information usage. Brickman, Gormally and Marchand Martella 
(2016) state that wide discrepancies exist between how academic staff and students, especially entry 
students, perceive feedback effectiveness. According to Brickman, Gormally and Marchand 
Martella, (2016) academic staff thought that feedback, including online platforms, helped student 
understanding and learning. Students strongly disagreed. Taylor (2008) concludes that timely 
feedback on early assessed work is a positive step in supporting student transition. Consequently, 
Brickman, Gormally and Marchand Martella (2016), like Brown, Roediger and McDaniel (2014), 
propose that written feedback specifically should be seen as an initiation of a post-performance 
discussion between academic staff and student, and part of a wider learning process. Rolfe, (2011) 
suggests that learning technologies such as Turnitin may offer a solution to support the transition 
into Higher education by providing accessible and rapid feedback online. Rolfe (2011) found that 
using online instant feedback, impacted positively on students’ cognition about their writing. This 
feedback, delivered predominantly through Turnitin, enabled students to develop their literacy 
skills and consider their writing styles within set texts. This is further corroborated by Schute (2008) 
who affirms that online technologies could support exciting and creative learning activities. 
Similarly, Brown, Roediger and McDaniel (2014) note that '[iJnterleaving and variation mix up the 
context of practice’ (p. 84) and leads to an enhancement of the learning process. In keeping with this, 
Lea and Street (2006) and Bhatia (2010) explore the connections between a text's purpose and its 
form, thereby interleaving a text's core subject and the linguistic and/or structural features, whilst 
noting that students have to mix these in their writing practice.In contrast, Chew, Jones and Blackey 
(2009) argue that the focal point should be pedagogically enabling and empowering students to learn 
with and through technology. To this end technology should be used to enhance learning, teaching 
and assessment activities. Buckley and Cowap (2003) suggests that technology enhanced learning 
research may lack an appreciation of different learning styles. The literature appears to focus on the 
troublesome or problematic areas of giving and interpreting feedback, rather than the reporting on 
effective strategies as well as where technology enhanced learning can support student engagement 
with feedback. 

Evans and Waring (2011), explored students' perceptions of feedback in relation to cognitive 
styles and culture, the study found cultural variables impact significantly on student assessment 
feedback preferences. The study encourages HEIs to consider the micro-cultures they promote when 
students are inducted into receiving feedback for learning. Using a social constructivist approach, 
Evans and Waring (2011), found that students are able to better consider their beliefs about learning 
which promotes ownership of learning and leads to self-regulation in learning. 

Evans (2016) has developed an Assessment Tool (EAT) which includes three core dimensions of 
practice: Assessment Literacy (requirements for assessment), Assessment Feedback (all feedback 
exchanges within an assessment), and Assessment Design (an integrated and meaningful approach 
to assessment which addresses: relevance, volume, inclusivity, collaboration, sustainability and 
manageability). The tool is grounded in the concept of students as active contributors to the 
assessment feedback process rather than seeing assessment as something which is done to them. 
Assessment practice is enhanced once the interconnected nature of the three core dimensions of 
practice is fully considered. The tool is fundamentally about promoting self-regulatory practice in 
assessment and asks "what does student engagement in assessment and feedback look like?" 
Software such as Turnitin, then, can potentially play a role in self-regulatory practices to enhance 
student performance, if designed correctly, and placed at the centre of a larger process of academic 
development through dialogue. 
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2. Method 
2.1. Research Design 


Using grounded theory and thematic coding in particular (Gibbs, 2007), we explored links between 
themes by relating responses back to theoretical perspectives. To further assist in the analysis of 
emerging themes, we found the framework suggested by Charmaz (2003) useful. This includes 
asking a series of basic questions during the intensive reading phase of the interview transcription: 
what is going on?; what are respondents doing or saying?; what do these actions and/or statements 
take for granted?; how do structure and context serve to support, maintain, impede or change these 
actions and/or statements? 

The richness of the responses made it possible to examine clusters of patterns and assign specific 
tags to better describe and conceptualise the narratives of the participants. During the decoding of 
the transcripts it became apparent that tensions exist between being a giving and receiving feedback 
online. The primary themes related to: meaning, purpose, action, and development of skills and 
competencies both as a member of staff and student. 

8 higher education professionals, and 14 students participated in group semi-structured 
interviews facilitated by the researchers. The practitioners and students responded to a general call 
for research participation. Students were undergraduates at various stages in their degree, and were 
drawn from Public Health and Sports Therapy courses. Partipication was entirely voluntary and 
there were no incentives offered. Likewise, practitioners were drawn from Public Health and 
Sports/Sports Therapy courses. 


2.2. Data Collection Tool 


Participants in the semi-structured group interviews were asked to discuss their experiences of using 
online platforms such as Turnitin for feedback and learning enhancement. These included both 
challenges and opportunities. All interview responses were recorded and transcribed. Participants 
were asked to review and check the transcripts for accuracy. All participants were made aware that 
participation in the study was voluntary and that they had a right to withdraw up until the point of 
data analysis. 


2.3. Data Analysis 


The current study took the form of a conventional content analysis. No preconceived categories were 
identified; instead, all categories and names for categories flowed directly from the data as observed 
(Kondracki & Wellman, 2002). Data collection and analysis focused exclusively on the narratives 
and comments emerging from the focus groups. Focus groups, using semi-structured interview 
questions were convened at different points during the academic year. The participants within each 
focus groups were students and staff studying and teaching, predominantly undergraduate 
students. Deliberately interleaving datasets, and responding to Brown et al. (2014), the present study 
merges quantitative and qualitative data from students with qualitative and quantitiative data from 
tutors, connecting these balanced approaches to previous studies that combine tutor and student 
data. Specifically, though many suggested improvements are guided by student opinion, our 
approach provides space for tutor dialogue. Participants were asked to discuss their experiences of 
receiving and/or giving feedback through online platforms such as Turnitin. They were asked about 
their understanding of the value and purpose of feedback, and how feedback could be used to 
enhance academic writing skills. The research protocol together with the semi-structured interview 
questions were reviewed and approved by the University of East London Research Ethics 
Committee. Comments, narratives, and recommendations ensuing from the interviews were then 
tabulated within an Excel spreadsheet for analysis. Both researchers read and reviewed the raw 
comments before deciding on relevant themes for analysis. 

Using Hermeneutic Phenomenology (Douglass & Moustakas, 1985), crafted stories (Crowther, 
Ironside, Spence & Smythe, 2017) and thematic coding in particular, (Gibbs, 2007) we explored links 
between themes by relating responses back to theoretical perspectives. This includes asking a series 
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of basic questions during the intensive reading phase of the interview transcription: what is going 
on?; what are respondents doing or saying?; what do these actions and/or statements take for 
granted?; how do structure and context serve to support, maintain, impede or change these actions 
and/or statements? As the participants described and narrated their individual and collective 
experiences of feedback, it was necessary to apply multiple tags and codes to the transcription of the 
interview narratives. 

The richness of the responses made it possible to examine clusters of patterns and assign specific 
tags to better describe and conceptualise the narratives of the participants. During the decoding of 
the transcripts it became apparent that tensions exist between staff and students, and these were 
further emphasised by a “connotation” category, which interpreted the results as either positive, 
negative, or neutral in import. The resultant categorised responses enabled some of these tensions 
and relationships to emerge more deeply in the semi-structured interviews. Here we used a form 
of template analysis as we coded transcripts, identifying themes. King (2004) describes template 
analysis as a set of techniques for thematically organising data. Some of the themes can be a priori 
though modified and interpreted by the researchers. We coded the same data independently, 
meeting to share our interpretations, reflect on the process, and develop our themes further. 


3. Results 


Focus group responses generated 398 total categorised comments. Of these, 111 were from tutors, 
and 289 comments were made by students. 


3.1. Connotation View 


Analysis of the comments demonstrate that - when considering the Turnitin platform - students 
and tutors appear to most frequently make mention of similarity scores/ plagiarism in combination 
with the general learning context in which that information is used. 


Connotation Per Response Category 
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40 
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Figure 1. Connotation per response category 
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The five most frequently recurring categories of response related to similarity scores, grades, the 
process of submission, and what feedback was for, or how a culture of feedback has emerged at the 
university. “Similarity” had a notable separation between positive, negative, and neutral, suggesting 
strong differences of opinion. 

The most frequently recurring useful categories of response are summarised below, in order of 
frequency. 


Similarity means discussions around the Feedback Studio similarity score, and how it may be used 
to detect similarity to the words used in other writers’ studies. There was a significant bifurcation of 
opinion in responses in this category, and it was clearly the most discussed concept. 


Feedback culture relates to debates concerning how tutors and students give, receive, and use 
feedback, and also how feedback software is used in the context of the university. There was a 
significant bifurcation of opinion in responses in this category, but fewer responses overall, and second only 
to “Similarity”. 

Grade: relates to discussions around academic scores and assessment grades in general. There were 
around twice as many positive responses with the “Grade” category than negative responses. 


Submission relates to the processes and conditions of submitting work for assessment, and can 
include practicalities such as deadlines. There were around four times more positive responses with the 
“Submission” comment than negative responses. 


Feedback for learning means how feedback can be used as part of the general learning cycle. 
Comments in this category focused specifically on the pedagogical potential of feedback, as distinct 
from the practicalities and institional culture of feedback (which is covered in the “feedback culture” 
category). 


Self-efficacy specifically relates to how students improve their own skills and learning by acting 
upon feedback. 


Academic integrity relates to discussions around the ethics of academic writing, most especially 
collusion, plagiarism, referencing/citation, and other related matters. This is different from 
“similarity”, which is focused on the similarity score statistic provided by Turnitin. 


Surface-level feedback means any consideration of features such as a punctuation, grammar, 
formatting, etc., which may be unrelated to the core arguments and thematic development of a text. 


Human vs. machine: comments in this category were concerned with whether automation has a 
place in the learning environment, and whether automation is valuable in enhancing learning. 


How tutors use it: comments in this category explored the ways tutors used the software. 
3.2. Most Notable Comments 


A comparison of some of the notable positive or negative comments amongst students and tutors 
appears in the table below. 

The range of responses shows notable bifurcation in both students’ and tutors’ assumptions as to 
whether and how the software benefits students as part of the formative learning process. None of 
the more notable responses from the frequent categories make mention of the act of writing. All of 
the notable responses above focus on the practicalities and purpose of the software. Many of these 
selected responses also address how far the software can be used to generate dialogue as part of the 
feedback process. 
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Table 1. 
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Notable positive and negative comments by frequent themes 


Category Notable positive response Notable negative response Related studies 
ey ; : — Ryan et al. (2009) and 
[Student:] ‘It build[s] you[r] confidence to write in your own word’ Piudens] So eene eee : pniulan toot Buckley and Cowap 
writers who I have never known (2013) 
iad 
sane aa: : : [Tutor:] ‘Students feel as though they're allowed todo Ryanet al. (2009) and 
[Tutor:] ‘is less of a worry for me. I've never had too many issues with ee ae : 
ineiadiaiaaneop eek Taknilear X' amount of plagiarism. [They think:] 'am I savvy Buckley and Cowap 
Plas P P ; enough to get under the limit?” (2013) 
oe : ; Ryan et al. (2009) and 
[Student:] ‘Lecturers feedback when you go through’ pd pomc teacher only give megane Buckley and Cowap 
feedback 
(2013)) 
F k 1’ Mai F maT : 
eedbac [Tutor:] ‘[Feedback Studio] does actually start to help us coach students eel ay py ey at Sieve ee my 
culture ls ; percentage figure, or am I on the right track. ... [think Ryan et al. (2009) and 
how to understand the criteria for their assessment and how they ; ; ae 
. the tool hasn't moved past that main function into one Buckley and Cowap 
should be able to self-regulate their own work before we've even ; wisi 
; f : ii? about general advice on essays; that's a very difficult | (2013) 
marked it to say "yes, I should be getting X for this. ene 
[Student:] ‘I go to the grade to check and go straight to the marking [Student:] ‘sometimes you lose marks if your Ryanced) ene 
; eae ay Buckley and Cowap 
band to strengthen my weakness. similarities are too high (2013) 
Grade [Tutor:] ‘That would be my biggest bugbear with 
[Tutor:] ‘We actually find that [when they have to guess their grades] Feedback Studio ... I have no control of Buckley and Cowap 
students grade themselves a lot more harshly’ [separating/suppressing the grade from the list of (2013) 
feedback], and I think that is a missed opportunity.’ 
oe . ; Turnitin (2016b); 
[Student:] ‘It guides me through my deadlines.’ Baden Tehelp watt dead ince nige tie Bandura (1997) and 
assessments. : 
os Pajares (2003) 
Submission Turnitin (2016b); 
ae [Tutor:] ‘If they submit again, I think it overrides the ine. yi 
[No positive tutor response. ] : : Bandura (1997) and 
first draft. : 
Pajares (2003) 
Padent| - ie aera omens . ea ator a anys ore [Student:] ‘It's very formal. It's a very formal way ... [I Shute (2008), Hattie 
as you've submitted it make an appointment with me and we'll go ; ; : ; ; 
derouanlat prefer to] get things explained [in person] & Timperely (2007) 
Feedback for & : , F 
\ ‘ [Tutor:] ‘I prefer to use multiple methods in feeding back to students. 
earning 


My students prefer audio. In this sense, they connect with my voice and 
understand the journey I travelled in reviewing and assessing the work 
- both the highs and the lows.’ 


[Tutor:] ‘Staff - we spent time feeding back without 
ever questioning for whom is the feedback intended.’ 


Shute (2008), Hattie 
& Timperely (2007) 
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3.3. Most Popular Keywords in Responses 


Analysis of the 10 most popular keywords from the 398 individual comments (excluding functional 
words such as ‘and’ or ‘like’) shows respondents notably mention feedback over three times more 
often than similarity: 


Table 2 
Frequency count of the most popular respondent keywords 
Keyword Grand Total 
feedback 108 
work 60 
think 56 
students 54. 
turnitin De 
similarity 34 
people 28 
comments 27 
need 26 


Common to these keywords are also notions of practicality, interaction, human communication 
and dialogue. There were 464 of these keywords overall. 


3.4. Relatedness of Responses to Surveyed Studies 


The studies by Ryan et al. (2009) and Buckley and Cowap (2013) were of particular relevance to the 
responses generated by the repsondents to the present study; these two predecessor studies were 
three times more pertinent than the second most relevant study. 


Number of Responses in Relation to Relevant Studies 
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Figure 2. Relatedness of Responses to Surveyed Studies 
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Discussion 


The present study sought to examine how software such as Turnitin functions to promote 
developmental learning. The literature suggests that feedback is a complex process often fraught 
with difficulty (Yorke, 2003). This complexity is heightened through a number of key factors, 
including purpose, direction, genre, language and impact. The results suggest that for feedback to 
be meaningful it needs to follow a developmental pathway, one that directs the receiver of the 
feedback to a functional outcome. Accordingly, the barriers to feedback for development are 
numerous, and often feedback intention is malaligned with feedback interpretation and value. 
Drawing on the work of Ryan et al. (2009), the study sought to better understand the tensions that 
exist between delivering feedback and receiving it. It was evident through the analysis that Turnitin 
can be a barrier to learning and thinking. This was captured in comments that tended to dehumanise 
feedback and present it as a mechanistic process devoid of interaction, critical discussion and 
collaboration. Hattie and Timperley (2007) make explicit that feedback has the potential to focus 
predominantly on remedial and corrective actions. Building on this, the present study found that for 
feedback to be useful to the recipient it needs to be anchored in and aligned with genre type. 
Moreover, the systemic approaches we foreground are intended to terminate in focused action 
points for learning development (Brown et al., 2014), rather than unfocused emotional reflection. 
Specifically, though many suggested improvements are guided by student opinion, our approach 
provides space for tutor dialogue. These findings raise important pedagogical as well as 
philosophical questions around the language used to feed back to students. The results further offer 
a critique around whether significant differentiation in feedback content and application needs to 
be consider against level of learning. Using a micro-macro-meso model, student and tutor responses 
suggest that the feedback cycle permeates individual and institutional domains. Online 
technologies, then, can potentially offer positive solutions for closing the expectation and experience 
gap. Students’ understanding of what is expected can differ vastly from staff views on the same 
issue. This study considered how students perceive writing expectations based on prior experiences. 
The results have identified a tension between using technology to avoid plagiarism and to promote 
meaningful learning development. 


Conclusion 


The findings suggest that feedback is complicated and can present multiple issues for both students 
and staff, as the bifurcation of opinion concerning both ‘similarity’ and ‘feedback culture’ goes some 
way to demonstrate. Even though the study could have benefitted from a cross-disciplinary 
approach, involving more participants, the initial results have implications for future software- 
based practices, and the Recommendations in our Appendix seek to present those within the general 
learning environment. 


References 


Amos, K., & McGowan, U. (2012). Integrating academic reading and writing skills development with core 
content in science and engineering. Journal of Learning Development in Higher Education. Retrieved 6 January 
2017 from 
http://www.aldinhe.ac.uk/ojs/index.php?journal=jldhe&page=article&op=download&path%5B%5D=1 


89&path%5B%5D=118b. 
Ball, S.J., Maguire, M., & Braun, A. (2012). How schools do policy, policy enactments in secondary schools. London: 


Routledge. 

Bandura, A. (1997). Self-efficacy and health behaviour. In A. Baum, S. Newman, J. Wienman, R. West, & C. 
McManus (Eds.), Cambridge handbook of psychology, health and medicine (pp. 160-162). Cambridge: Cambridge 
University Press. 


E. D. Abrahamson, J. Mann / Journal of Pedagogical Research, 2(3), 2018, 145-166 155 


Bhatia, V.K., (2010). Interdiscursivity in professional communication. Discourse and Communciation, 4 (1), 32- 
50. 

Black, P., & William, D. (1998). Assessment and classroom learning. Assessment in Education, 5, 7-74. 

Braine, G. (1997). Beyond word processing: networked computers in ESL writing classes. Computers and 
Composition, 14(1), 45-58. 

Brickman, P., Gormally, C., & Marchand Martella, A. (2016). Making the Grade: Using instructional feedback 
and evaluation to inspire evidence-based teaching. CBE Life Science Education, 15 (4). doi:10.1187/cbe.15- 
12-0249 

Brown, P., Roediger, H., & McDaniel, M. (2014). Making it stick. Harvard University Press. 

Buckley, E. & Cowap, L. (2013). An evaluation of the use of Turnitin for electronic submission and marking 
and as a formative feedback tool from an educator's perspective. British Journal of Education Technology, 44 
(4), 562-570. 

Butler A. C., & Roediger, H. (2008). Feedback enhances the positive effects and reduces the negative effects of 
multiple-choice testing. Memory and Cognition, 36, 604-616. 

Charmaz, K. (2003). Grounded theory: Objectivist and constructivist methods. In N. K. Denzin, & Y.S. Lincoln 
(Eds.), Strategies for qualitative inquiry (2° Ed) (pp. 249-291). Thousand Oaks, CA: Sage. 

Chew, E., Jones, N., & Blackey, H. (2009). A UK Case Study - Technology enhances educational experiences 
in the University of Glamorgan. Future Computer and Communication 2009, 212-216. 

Cho, K. & Schunn, C. D. (2007). Scaffolded writing and rewriting in the discipline: a web-based reciprocal peer 
review system. Computers & Education, 48, 409-426. 

Coffey, S. & Anyinam, C. (2012). Trialing a contextual approach to academic honesty. Nurse Education, 37(2), 
62-66. 

Crowther, S., Ironside, P, Spence, D., & Smythe, L. (2017). Crafting stories in Hermeneutic Phenomenology 
Research: A methodological device. Qualitative Health Research, 27 (6), 826-835. 

Department for Business Innovation and Skills [BIS] (2016a). Teaching Excellence Framework: Technical 
Consultation for Year Two. Retrieved 22 August 2016 from 
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/523340/bis-16-262-teaching- 
excellence-framework-techcon.pdf. 


Department for Business Innovation and Skills [BIS] (2016b). Success as a Knowledge Economy: Teaching 
Excellence, Social Mobility and Student Choice. Retrieved 4 January 2017 from 
https://www.dropbox.com/home/research/ Academic %20Literacy %20Articles/TEF?preview=bis-16- 


265-success-as-a-knowledge-economy+t-+Jot+tJohnson+Report+May+2016.pdf. 

Deane, P., Odendahl, N., Quinlan, T., Fowles, M., Welsh, C., & Bivens-Tatum, J. (2008). Cognitive models of 
writing: Writing proficiency as a complex integrated skill. ETS RR-08-55. Retrieved 21st July 2017 from 
https://www.ets.org/Media/Research/pdf/ RR-08-55.pdf. 

Douglass, B., & Moustalcas, C. (1985). Heuristic inquiry: The internal search to know. Journal of Humanistic 
Psychology, 25(3), 39-55. 

Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., & Japel, C. (2007). 
School readiness and later achievement. Developmental Psychology, 43(6), 1428-1446. 

Evans, C. & Waring, M. (2011). Student teacher assessment feedback preferences: The influence of cognitive 
styles and gender. Learning and Individual Differences, 21(3), 271-280. 

Evans, C. (2016). Enhancing assessment feedback practice in higher education: The EAT framework. Retrieved 17 


September 2017 from _ https://www.southampton.ac.uk/assets/imported/transforms/content- 


block/UsefulDownloads_Download/ A0999D3AF2AF4C5A A24B5BEA08C61D8E/EAT%20Guide%20Ap 
ril%20FINAL1%20ALL. pdf. 


Gibbs, G.R., (2007). Analyzing qualitative data. London: Sage. 

Hattie, J. & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77 (1), 81-112. 

Hepplestone, S., Holden, G., Irwin, B., Parkin, H.J., & Thorpe, L. (2011). Using technology to encourage student 
engagement with feedback: a literature review. Research in Learning Technology, 19 (2), 117-127. 

Jeffries, R. (2001). Essential XP: Card, Conversation, Confirmation. Retrieved 25 March 2017 from 
https: / /ronjeffries.com/xprog/articles/expcardconversationconfirmation. 

King, N. (2004). Using templates in the thematic analysis of text. In C. Cassell, & G. Symon (Eds.), Essential 
guide to qualitative methods in organizational research (pp. 256-270). London: Sage. 

Knight, A. (2004). The collaborative method. A strategy for improving Australian general practice. Australian 
Family Physician, 33(4), 269-274. 

Kondracki, N.L. & Wellman, N.S. (2002). Content analysis: Review of methods and their applications in 
nutrition education. Journal of Nutrition Education and Behavior, 34(4), 224-230. 


156 E. D. Abrahamson, J. Mann / Journal of Pedagogical Research, 2(3), 2018, 145-166 


Kostka, I., & Malibroska, V. (2016). Using Turnitin to provide feedback on L2 Writers’ Texts. TESL-E]J, 20(2), 1- 
22. 

Lavelle, E. & Zuercher, N., (2001). The Writing Approaches of University Students. Higher Education, 42 (3), 
373-391. 

Lea, M. & Street, B. (2006). The “Academic Literacies” model: Theory and applications. Theory into Practice, 45 
(4), 368-377. 

Liou, H. C. & Peng, Z. Y. (2009). Training effects on computer-mediated peer review. System, 37, 514-525. 

Lundstrom, K. & Baker, W. (2009). To give is better than to receive: the benefits of peer review to the reviewer's 
own writing. Journal of Second Language Writing, 18, 30-43. 

Mann, J. (2016) Using Turnitin to improve academic writing: an action research inquiry. Research in Teacher 
Education, 6(2), 16-22. 

Nattinger, J. & DeCarrico, S. (1992). Lexical phrases and language teaching. Oxford: Oxford University Press. 

Pajares, F. (2003). Self-efficacy beliefs, motivation, and achievement in writing: A review of the literature. 
Reading and Writing Quarterly, 19, 139-158. 

Paulus, T. M. (1999). The effect of peer and teacher feedback on student writing. Journal of Second Language 
Writing, 8 (3), 265-289. 

Rolfe, V. (2011), Can Turnitin be used to provide instant formative feedback? British Journal of Educational 
Technology, 42, 701-710. doi:10.1111/j.1467-8535.2010.01091. 

Ryan, G., Bonanno, M. A., Krass, I., Scouller, K., & Smith, L. (2009). Undergraduate and postgraduate 
pharmacy students' perceptions of plagiarism and academic honesty. American Journal of Pharmaceutical 
Education, 73(6), 1-8. 

Saint, D., Horton, D., Yool, A., and Elliott, A (2015). A progressive assessment strategy improves student 
learning and perceived course quality in undergraduate physiology. Advances in Physiology Education, 39(3), 
218 - 222. 

Scheeler, M. C., McKinnon, K., & Stout, J. (2012). Effects of immediate feedback delivered via webcam and 
bug-in-ear technology on preservice teacher performance. Teacher Education and Special Education, 35 (1), 
77-90. doi: http:/ /dx.doi.org/10.1177/0888406411401919. 

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78 (1), 153-189. 

Storch, N. (2005). Collaborative writing: product, process, and students’ reflections. Journal of Second Language 
Writing, 14, 153-173. 

Swales, J. (1990). Genre analysis: English in academic and research settings. Cambridge, England: Cambridge 
University Press. 

Taras, M. (2003). To feedback or not to feedback in student self-assessment. Assessment and Evaluation in Higher 
Education, 28 (5), 549-565. 

Taylor, E. W. (2008), Transformative learning theory. New Directions for Adult and Continuing Education, 119, 5- 
15. doi:10.1002/ace.301 

Van der Hulst, J. van Boxel, P & Meeder, S. (2014). Digitalizing feedback: reducing teachers’ time investment 
while maintaining feedback quality. Retrieved 6 April 2016 from http:/ /media.leidenuniv.nl/legacy/ecel- 
paper-jvdhulst-pvboxel-def.pdf. 

Wingate, U., Andon, N., & Cogo, A. (2011). Embedding academic writing instruction into subject teaching: A 
case study. Active Learning in Higher Education, 12 (1), 69 -81. 

Yang, Y. (2011). A reciprocal peer review system to support college students’ writing. British Journal of 
Educational Technology, 42(4), 687-700. 

Yorke, M. (2003). Formative assessment in higher education: moves towards theory and the enhancement of 
pedagogic practice. High Education, 45, 477-501. 


E. D. Abrahamson, J. Mann / Journal of Pedagogical Research, 2(3), 2018, 145-166 157 


Appendix. Recommended software changes for enhancing feedback 


By relating key comments from the interviews in our study to the work of Yorke (2003) and Bhatia (2010), key differentials such as user type, 
descriptions change request and anticipated outcomes can be identified, following the user story software framework advocated by Jeffries (2001). 
These differentials have implications for software development and revision to the current Turnitin platform, enabling staff and students to better 
consider the value, purpose and impact of feedback as a change agent. Building on the comments of both students and tutors, it is possible that changes 
to software can effect changes to feedback culture. These recommendations are presented in the table below. 


Full user story 


Related Notable Focus Group Comment 


Related frequent 
comment category 


Related Study/Theme 


As a student, I want to see my tutor's feedback as 
bubbles around the text, which I can read easily, 
expand, and copy as I see fit, so that I can easily 
navigate my way around the feedback and 
understand it fully. This change would affect the 
front-end feedback entry area. 


[Student:] Staff need to spend more time, considering for whom the 
feedback is intended. I often feel staff are feeding back to themselves. 


Feedback that is wordy and complicated is a distractor not an enabler 
for learning. 


some teachers only give negative feedback 


how tutors use it; 
feedback culture; 
feedback for learning 


Structured feedback for 
assessment 
improvements; 
additional resources to 
support this - Wingate 
et al. (2011) 


As a university, I want to introduce my own 
corporate branding, contact details, etc., onto the 
system, so that I can present the system to my 
students as a seamless part of the overall service 
they receive from me; this will help me promote 
the service fully. This change would affect the 
entire system, or multiple areas. 


[Tutor:] It's not fulfilling that feedback loop because not enough 
students are accessing it. 


how tutors use it; 
feedback culture; 
feedback for learning; 
human vs. machine 


knowledge economy - 
BIS (2016b); Mann 
(2016) 


As a university, I want to host the service on my 
own domain, such as: 

http:/ /tutorfeedbacksystem.myinstitution.ac.uk, 
so that I can present the system to my students as a 
seamless part of the overall service they receive 
from me; this will help me promote the service 
fully. This change would affect the entire system, 
or multiple areas. 


[Tutor:] It's not fulfilling that feedback loop because not enough 
students are accessing it. 


how tutors use it; 
feedback culture; 
feedback for learning; 
human vs. machine 


knowledge economy - 
BIS (2016b) ; Mann 
(2016) 
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As a student, I want to receive feedback that 
is more about my development as an 
academic writer in my chosen discipline 
than it is about a punitive approach to 
similarity and plagiarism, so that I can 
develop, progress, and succeed as a learner. 
This change would affect the front-end 
feedback entry area. 


[Student:] even if you have perfect writing ... you can lose grades 
[by having high levels of similarities] 


more information about my weaknesses, about what is missing 
from my essay 


I had about two paragraphs, I forgot to change in my own 
understanding. That was a wake up call. 


[give] feedback [that identifies] where you need to improve 
they need to not believe Turnitin too much 
feedback show|[s] where to improve 


[Tutor:] It's not fulfilling that feedback loop because not enough 
students are accessing it. 


My view would be that is Turnitin a plagiarism tool or is it a 
development tool? 


There should be a little bit of coaching or feedback within it, not 
just the percentage [similarity score]. 


[Student:] Whoever owns [the] institution ... if they bring in like 
Turnitin, they're saving time and money but they're also limiting 
on what we do and what feedback we can get. 


At an earlier level, I think students probably need more guidance. 


feedback culture; feedback for 
learning; self-efficacy; surface- 
level feedback; similarity; 
academic integrity 


Structured feedback 
for assessment 
improvements; 
additional resources to 
support this - Wingate 
et al. (2011); impactful 
nature of 
developmental 
feedback - Hattie and 
Timperely (2007); 
Mann (2016) 


As an administrator, I want to create lots of 
bespoke overrides and permissions about 
nearly all of the features of the software, 
based on an institutional, full-administrator, 
tutor, or student profile, so that I can toggle 
the level of support that is available to all 
user types. This change would affect the 
entire system, or multiple areas. 


Whoever owns institution ... if they bring in like Turnitin, they're 
saving time and money but they're also limiting on what we do 
and what feedback we can get. 


human vs. machine; feedback 
culture; feedback for learning; 
self-efficacy; how tutors use it 


Ensuring all 
protagonists are all 
actively involved in 
the feedback process - 
Taras (2003); 
knowledge economy - 
BIS (2016b) 
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As an administrator, I want to integrate the 
system with other data-producing systems such 
as Turnitin, Moodle, Blackboard, Mahara, SITS, 
etc.; this can be linked to the tutor feedback 
functions in a seamless and entirely 
customisable way; I can also decide what user 
type can see what information, and how the 
information is displayed, so that I can provide 
students and tutors alike with a completely 
customisable view of performance data linked 
to the feedback provided. This change would 
affect the system database. 


if Isubmitted my work and I got it back at 40[% similarity]; if 
I hadn't seen them [the tutor] I could have got a 50[% 
similarity]. When you go to see them [the tutors] you can 
push it up to like a first because you get the clarification. It 
sinks more into your head if you're communicating. 


how tutors use it; feedback 
culture; feedback for learning; 
similarity; self-efficacy; grade; 
academic integrity 


Importance of seeing 
feedback within the 
context of other 
academic activities - 
Duncan et.al. (2007) 


As a tutor, I want to have an option where 
tutors can receive qualitative and quantitative 
feedback from the student, which assesses and 
comments upon the feedback that the tutor 
originally made, so that I can make the 
feedback process truly circular, so as to 
understand how to constantly improve my own 
feedback practices. This change would affect 
the front-end feedback entry area. 


[Student:] I really don't [always] know where I went wrong, 
so I'm going to have to just fix up on my sentence structures. 
Make my own feedback, basically. 


self-efficacy; feedback for 
learning; how tutors use it; 
surface-level feedback 


Technology used in the 
feedback cycle - Ball, 
Maguire, & Braun, 
(2012); Mann (2016) 


As a student, I want to view a statistic based on 
how other people on your course have done in 
the same assignment. This is a single average 
number of other students’ grades. From this, I 
can see whether I am averaging higher or lower 
marks, so that I can see how well Iam doing, 
and how I can get even better. This change 
would affect the front-end feedback entry area. 


[Student:] ... | was aware of it [needing to improve] and that's 
because it was a draft for us to see where we are standing and 
to see where we can improve it 


grade; feedback for learning; self- 
efficacy; submission; academic 


integrity 


Motivational aspects of 
understanding peer 
performance - 
Hepplestone et.al. 
(2011); Yang (2011) 


As a student, I want to check other students’ 
ratings of the feedback they have received. This 
feature allows me to see the general rating that 
has been provided by your peer group, so that I 
can check my feedback-ratings and opinions of 
tutors' feedback against those of my peergroup. 
This change would affect the front-end 
feedback entry area. 


[Student:] We're not asking you to do our essays for us, but to 
say 'this is really good' and 'maybe you should expand on 
this', 'maybe you should look at this'. Just some more 
guidance. 


feedback culture; feedback for 
learning; self-efficacy; how tutors 
use it 


Students' perceptions 
of feedback - Evans 
and Waring (2011) 
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As a student, I want to receive structured feedback on 
my academic work, so that I can be more successful in 
my assignments at university. This change would 
affect the entire system, or multiple areas. 


[Student:] ... I was aware of it [needing to improve] and that's 
because it was a draft for us to see where we are standing and to 
see where we can improve it 


probably the more you work ... the more you develop your 
writing skills ... when my colleagues asked me, I was [better able 
to do the work]. It gives me more confidence. ... when I 
submitted my work, it didn’t show any kind of plagiarism or 
anything. It just gave me a good feeling that I had done good 
work on my own.... It increases your self confidence. ... I feel I can 
do it, whereas before I felt I couldn't. ... | was scared at the 
beginning, but then once I took more steps I got there. 


[Tutor:] But I haven't ever said "do you want feedback on the 
content or do you want feedback on the style?" 


feedback for learning; feedback 
culture; submission; human vs. 
machine; how tutors use it; self- 
efficacy; surface-level feedback; 
academic integrity 


structured feedback 
mechanisms - Yorke 
(2003); Butler and 
Roediger (2008) 


As an administrator, I want to generally control the 
many layers of the software's functions, and create 
bespoke services based on my preferences, tutors' 
preferences, institutional preferences, or students’ 
preferences, so that I can enjoy a system that is 
completely customised for my institution. This change 
would affect the entire system, or multiple areas. 


Whoever owns institution ... if they bring in like Turnitin, they're 
saving time and money but they're also limiting on what we do 
and what feedback we can get. 


human vs. machine; self- 
efficacy; feedback culture; 
feedback for learning 


Ensuring all 
protagonists are all 
actively involved in the 
feedback process - Taras 
(2003); knowledge 
economy - BIS (2016b) 


As a student, I want to view advice on the structural 
features, language, usual conventions, and typical 
numerical functions of the kind of text Iam writing, 
prior to handing it in. There should be some sort of 
lookup process that matches the text-type to exemplar 
material for me to review, so that I can better meet the 


[Student:] more information about my weaknesses, about what is 
missing from my essay 


when I wrote my work in Microsoft Word I saw I was having 
grammar problems but when I looked on Turnitin it looked more 
professional so I could pick up on [things like] sentence structure 


surface-level feedback; feedback 
for learning; submission; self- 
efficacy; feedback culture 


feedback on adopting 
peers' stylistic features - 
Yang (2011); feedback 
on structural features of 
a text - Amos and 


purposes of my text. This change would affect the ... and change my work ... it makes you want to make you get McGowan (2012); 
front-end feedback entry area. your work looking more professional 

; [Student:] I think with our class in particular, some people have 
fee Sen, LWemit De abe to View an ari ple tet taken a lot of time out of education before they have come to uni. students think feedback 


that is similar in purpose and content to the one I have 
to write, but is not an exact match, so that I can 
understand how other people write these sorts of texts 
successfully, and easily access it through the 
submission/ feedback window. This change would 
affect the front-end feedback entry area. 


I've been out of education years before uni, and ... before this, 
well, I hadn't [written] a paragraph in God knows how many 
years. It's a massive step. 


[Student:] I would prefer directed feedback to provide illustrated 
examples of how to improve my work 


self-efficacy; feedback for 
learning; surface-level feedback; 
how tutors use it 


is better when it 
includes specific 
examples - Scheeler, 
McKinnon and Stous, 
(2012) 
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As a tutor, I want to be able to upload a text that I 
have decided is a suitable exemplar of the kind of 
text I expect students to write. This text may be a 
previously-published work, or an example froma 
previous student cohort, so that I can help students 
better understand the purpose, audience, and 
techniques of these texts. This change would affect 
the front-end feedback entry area. 


[Student:] ... | was aware of it [needing to improve] and that's 
because it was a draft for us to see where we are standing and 
to see where we can improve it 


feedback for learning; 
feedback culture; submission; 
human vs. machine; how 
tutors use it; self-efficacy; 
surface-level feedback; 
academic integrity 


students think 
feedback is better 
when it includes 
specific examples - 
Scheeler et.al. (2012) 


As a student, I want to check feedback from my 
most recent work. I can also see a quick summary 
of feedback from work before that. This can be 
filtered by module, tutor, or time., or academic 
year, so that I can understand what I need to work 
on, and provide a useful report of how my skills 
have changed/improved over time. This change 
would affect the front-end feedback entry area. 


[Student:] probably the more you work ... the more you 
develop your writing skills ... when my colleagues asked me, 
I was [better able to do the work]. It gives me more 
confidence. ... when I submitted my work, it didn’t show any 
kind of plagiarism or anything. It just gave me a good feeling 
that I had done good work on my own... It increases your self 
confidence. ... I feel I can do it, whereas before I felt I couldn't. 
... [was scared at the beginning, but then once I took more 
steps I got there. 


feedback culture; feedback for 
learning; self-efficacy; 
submission 


Feedback and 
progression - BIS 
(2016a); careful 
feedback cycles can 
improve progression - 
Wingate et al. (2011) 


As a student, I want to be able to see a report that 
shows short extracts of my previous work that had 
a certain theme of feedback; these can be shown in 
a row-by-row list, or through some form of time 
display, so that I can see how far my practice has 
developed in certain key areas related to the 
features of specific genres of text I have had to 
write in the past. This change would affect the 
front-end reports. 


[Student:] I have noticed that even though I have I have put a 
lot of effort in I only get like one sentence back in feedback ... 
to me I don't think that gives you room to improve. [I'd need 

to have] a few more sentences telling me exactly what it went 
wrong like which paragraph was it which line was it. 


how tutors use it; feedback 
culture; feedback for learning; 
self-efficacy; surface-level 
feedback 


Self-efficacy (Bandora); 
careful feedback cycles 
can improve 
progression - Wingate 
et al. (2011) 


As a student, I want to read a report as to when I 
handed in my work for feedback, so that I can help 
students track the current progress of their 
enquiries, and manage their deadlines more 
efficiently, thereby further enabling efffective 
progress. This change would affect the front-end 
reports. 


[Student:] I think the way to improve the feedback process 
would be to adopt an Amazon type approach whereby 
students are kept informed about status of the submission 
and when grades and feedback will be released. I often find I 
submit work and then forget about the content until I receive 
the work back. I rarely act on the feedback given. 


Yes once you learn how to upload and submit, that's it. It is 
easy hence I am confident. 


It's easy to go back and read it again without losing 
information. 


human vs. machine; feedback 
culture; submission 


Feedback as an 
ongoing 
developmental process 
- Hattie and Timperley 
(2007, in van der Hulst 
et al., 2014) and Shute 
(2008), in van der 
Hulst et al., 2014). 
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As an administrator, I want to decide which 
reports some tutors, or all tutors, are able to see, 
and whether I want to make my feedback 
information visible to other institutions, so that I 
can take control over the information passed to 
tutors, and possibly contribute to a cross- 
institutional network of progression data. This 
change would affect the entire system, or multiple 
areas. 


[Tutor:] there were some weaknesses in terms of accessing the 
primary resources. You know when you go to on the right 
hand side and click on it and you have to request permission 
from the other institution. That's a bit annoying I suppose. 


how tutors use it; feedback 
culture 


Feedback defined as 
developmental 
information provided 
by an agent - Hattie 
and Timperley (2007); 
knowledge economy - 
BIS (2016b) 


As a student, I want to open up a text/audio chat 
and/or video window to speak to an online tutor 
in an ad-hoc manner, so that I can receive real-time 
tutoring and/or reflection on my recent feedback, 
possibly in addition to the feedback I get through 
an assignment submission. This change would 
affect the entire system, or multiple areas. 


[Student:] However, if I said that we are going to spend time 
[with the tutor] individually to point out where I went wrong. 
This is [software] is just the beginning 


We should get time though [for human contact] ... if you look 
at the student as a consumer, we pay a lot of money. We don't 
want everything to be online . We want to talk to people face 
to face. We pay a lot of money: we should have a lot of one- 
on-one [contact tutorial time]. 


how tutors use it; feedback 
culture; human vs. machine; 
feedback for learning 


Digitals can make mistakes that humans can spot. 


Dehumanised. 


Discussions of recent 
feedback to enrich the 
feedback learning 
process - Brickman 
et.al (2016); knowledge 
economy - BIS (2016b) 


As a tutor, I want to record and upload generic 
video tutorials, which are categorised on the 
system according the overall theme of the tutorial, 
which can be linked to the "quick tutorial" 
comments, so that I can provide generic help for 
students. This change would affect the entire 
system, or multiple areas. 


Turnitin does encourage people to give quite a lot of 
comment about grammar and writing and how to do the 
essay but you could just develop a Quick Mark set about 
what is more specific to your topic and what is the feedback 
for your learning outcomes... You have to focus on what your 
goals are and make sure that's what you're giving them 
feedback on. 


feedback for learning; 
submission; grade; feedback 
culture; human vs. machine 


Providing feedback 
that is linked to 
academic writing 
genres more than 
plagiarism 
performance or just 
functions such as 
grammar - Coffey & 
Anyinam (2012), 
Buckley and Cowap 


(2013 
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As a student, I want to search for and view 
individual pre-recorded tutorial videos that 
are related to my current enquiry; perhaps 
receive suggestions based on the text of the 
essay | have just submitted, so that I can 
receive specific and targeted help on areas of 
development for my academic writing. This 
change would affect the entire system, or 
multiple areas. 


However, if I said that we are going to spend time [with the tutor] 
individually to point out where I went wrong. This is [software] is just the 
beginning. 


I just think we should have more class time with [tutors]. When [my tutor] 
puts his YouTube videos up before the class and expects us to watch them 
before the class, I think that's really good. 


I don't think [Turnitin] is as much as a help as these [academic writing 
tutorial] sessions. Before today['s academic writing session], I was 
wondering 'how am I going to write this essay?'. I think that's [i.e., 
academic writing tutorial is] more helpful than Turnitin. 


feedback for learning; human 
vs. machine; feedback culture; 
self-efficacy; surface-level 
feedback 


Student can effect 
change to their learning 
by identifying their 
developmental needs - 
Bandura (1997) and 
Pajares (2003) in Kostka 
and Malibroska (2016) 


As a tutor, I want to be able to see how 
productive I am in comparison with other 
tutors - i.e., what my average turnaround 


Timely and specific 


feedback enh: 
time is for dealing with the work I have [Tutor:] [There is this] culture of submitting to the deadline and so the submission; grade; feedback ora i ana 
i oo fea fae : 4 students' learning - 
received, so that I can check my productivity early submission to Turnitin is rare, I think. culture; self-efficacy . 
; ; iaeor Hattie and Temperley 
against others and use this as a motivating (2007) 
tool. This change would affect the entire 
system, or multiple areas. 
[Student:] I think with our class in particular, some people have taken a lot 
of time out of education before they have come to uni. I've been out of 
education years before uni, and ... before this, well, I hadn't [written] a 
paragraph in God knows how many years. It's a massive step. 
Closing the gap between 


As a tutor, I want to be able to edit a series of 
predetermined "quick tutorial" functions, 
that list common errors, and explain them in 
simple, non-technical language that the 
student will be able to understand, so that I 
can help students understand exactly how to 
succeed at their work, and not be baffled by 
unnecessary technical/linguistic 
terminology. This change would affect the 
front-end feedback entry area. 


[Student:] I had a set of 60 comments that I used to use, and put in 3 - 4 by 
hand, and then I transposed that into Turnitin and now I have different 
sets for each assignment. ... I'm more about thr content of it. 


[Tutor:] Turnitin does encourage people to give quite a lot of comment 
about grammar and writing and how to do the essay but you could just 
develop a Quick Mark set about what is more specific to your topic and 
what is the feedback for your learning outcomes... You have to focus on 
what your goals are and make sure that's what you're giving them 
feedback on. 


I like the idea that you're building a Quick Mark set for that assignment. 
It's not "here's the bank of 5 million quick marks that just focus on writing 
skills. 


feedback culture; feedback for 
learning; human vs. machine; 
how tutors use it; surface-level 
feedback 


standard achieved and 
standard required - 
Brickman et.al (2016); 
Knight (2004), Rolfe 
(2011) - online 
technology to support 
individual learning; 
specific feedback is 
useful for learning - 
Scheeler et.al. (2012) 
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As an administrator, I want to be able to 
decide whether tutors, or specific tutors, 
are able to edit a series of predetermined 
"quick tutorial" functions, that list 


Closing the gap 
between standard 
achieved and standard 
required - Brickman 


mmon errors, and explain them in Tutor:] Th nd thing I most fr ntly i k i 
seas anti Mane te genie : Luton) The seco ae na nae Hee Bespoke Ouick feedback culture; feedback for et.al (2016); Knight 
simple, non-technical language that the Mark set that I generated before we even used this. I had a set of 60 : : 
; : learning; human vs. machine; —_ (2004), Rolfe (2011) - 
student will be able to understand, so that | comments that I used to use, and put in 3 - 4 by hand, and then I : : 
: a : how tutors use it; surface- online technology to 
Ican help students understand exactly transposed that into Turnitin and now I have different sets for each fo gies 
; : : level feedback support individual 
how to succeed at their work, and not be assignment. ... I'm more about the content of it. : “ps 
learning; specific 
baffled by unnecessary 


feedback is useful for 
learning - Scheeler 
et.al. (2012) 


technical/linguistic terminology. This 
change would affect the front-end 
feedback entry area. 


As a student, I want to know when my 


feedback is due to be provided to me [Student:] I think the way to improve the feedback process would be 


to adopt an Amazon type approach whereby students are kept Henclyand eaee 


(which can take the form of an attractive : ee feedback enhances 
: a : informed about status of the submission and when grades and ae ; : 
progress" graphic), so that I can plan my ; ; . submission; feedback culture _ students' learning - 
: : : : feedback will be released. I often find I submit work and then forget : 
work around it and hit my deadlines. This : Hattie and Temperley 
about the content until I receive the work back. I rarely act on the 
change would affect the front-end . (2007) 
feedback given. 
feedback entry area. 
As a student, I want to be able to read [Student:] I would rather pay for somebody to read my essay and go feedback culture; feedback for Timely and specific 
basic feedback within about 24 hours, so through it with me than submit it to Turnitin and be givena feedback —_ learning; humanvs. machine; feedback enhances 
that Ican plan my work around itand hit then. how tutors use it; surface- students' learning - 
my deadlines. This change would affect level feedback; self-efficacy; Hattie and Temperley 
the entire system, or multiple areas. ‘students as a consumer' submission; feedback culture (2007) 
As a tutor, I want to be able tosee a quick = [Tutor:] The trouble with a prior submission to Turnitin to check it is Timely and fi 
tally of all the papers I have examined in the assumption that they're not working to the deadline, and there I se or: Dee ani 
: ; ‘ feedback culture; submission; feedback enhances 
the last X days/ Y hours/ Z minutes, so think you've got a huge issue because most of our students work to the : : 

: 4 ; ; self-efficacy; feedback for students' learning - 
that I can understand my own deadline. ... I will have students in my class at 9am in the morning eee Hiaihe and Temerle 
productivity. This change would affect the | who still perceive that they have enough time to complete the 6 (2007) penny 
entire system, or multiple areas. assignment that's due that day. 

As a student, I want to be able to see : ps 
ae ; Timely and specific 
statistics about my tutor's usual 
: feedback enhances 
turnaround time, and what other people ; ‘ i ; ; : 
sq : [Student:] If we're not happy with the products for which we're how tutors use it; feedback students' learning - 
have said in that regard, so that I can, if : : i : 
ae paying, usually you take it back, don't you? culture; human vs. machine Hattie and Temperley 
the system permits it, choose my tutor. 
(2007); knowledge 


This change would affect the entire 


system, or multiple areas. economy - BIS (2016b) 
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As an administrator, I want to be able to integrate the data and reporting 
functions from Turnitin, Moodle, Blackboard, etc., into the system. This 
way, I can see a history of my previous feedback information, and see 
how it is related to my previous grades. If my institution uses, for 
example, Turnitin, then submitting to Turnitin should somehow feel 
seamless, and should seem to be part of the feedback/text examplar 
system, so that I can students and tutors alike are able to use multiple 
systems at once without having to log off or switch tabs in their 
browsers, and see how the feedback they have received on this system 
potentially relates to their performance on others. This change would 
affect the multiple areas. 

As a tutor, I want to be able to highlight any area of a student's text, and 
label it in accordance with a progress score marker from their previous 
academic work relating to the conventions of that academic writing 
genre, so that I can ensure that students can, over time, identify how they 
have developed specific aspects of their writing practice. This change 
would affect the multiple areas. 


[Tutor:] Do they evaluate themselves? 
If they've got to go through and think 
what do I think my grade is... the idea 
there would be they've got feedback, 
you would judge it against this this and 
this, and this is what you've done, and 
what do you think you've been graded? 
To try and make them get a realistic 
appreciation of their work, and kind of 
process the feedback independent of 
the grade. 


[Student:] I just feel like we're owed 
more feedback. I feel like it should be 
taken more in depth 


feedback culture; 
feedback for learning 


feedback culture; 
feedback for learning; 
human vs. machine 


Linking feedback to the assessment and 
learning cycle, as an assessment tool - 
Evans (2016); structured feedback in the 
context progressive approaches to 
assessment - Yorke (2003); Butler and 
Roedige(2008) 


Semi-automated feedback as part of a 
progression cycle - Yang (2011); feedback 
as formative process - Williams (1998); 
revision of writing as part of feedback 
cycle - Deane et al. (2008); knowledge 
economy - BIS (2016b) 


As a student, I want to be able to see machine-generated statistics on the 
rates of error to do with grammar and punctuation, so that I can 
understand the rate of basic errors in my text, and gain some 
understanding of where I can improve or have improved. This change 
would affect the front-end feedback entry area. 


[Student:] I don't think we've got it yet, 
but there is a sort of a grammar 
checking tool that you could get ... so I 
think that that could be beneficial on 
certain levels as well 


surface-level feedback; 
human vs. machine 


Possible use of semi-automated 
technology to promote progression of 
student learning - Wingate et al. 
(2011);Yang (2011) 


As an administrator, I want to toggle the settings as to whether a student 
can see all available tutors and decide who to send it to, or whether to 
make the system a "black box" where tutors' details are blocked from 
students' front-end interfaces, so that I can add a layer of choice to the 
process. This change would affect the entire system, or multiple areas. 


[Student:] The major theme [for me] is 
about human interaction. ... all this 
technology isn't preparing us for when 
we graduate. 


human vs. machine 


feedback process is most effective when 
all the protagonists are actively involved 
- Taras (2003) 


As a student, I want to upload a photo, a short biography, and 
information concerning the course I am studying, and the subjects I am 
most interested in - as well as some description of my general long or 
short-term goals for my education and career - so that I can help my 
tutors fully understand my particular needs. This change would affect 
the account changes area. 


[Turnitin] is just so dehumanising 


human vs. machine; 
feedback culture 


pedagogically enabling students and 
empowering students - Chew and Jones 
(2009); connecting feedback to personal 
goals as pedagogically valuable - Rolfe 
(2011); Saint, et.al., (2015) 


As a tutor, I want to, upon receiving work from a student, see a student's 
course information (if they supply it), a quick history of their account, 
their previous work, their previous error rates, and their previous 
feedback of other tutors, so that I can provide students with a tailored 
response to their work. This change would affect the account changes 
area. 


[Turnitin] is just so dehumanising 


human vs. machine; 
feedback culture; 
feedback for learning; 
self-efficacy; how 


pedagogically enabling students and 
empowering students - Chew and Jones 
(2009); connecting feedback to personal 
goals as pedagogically valuable - Rolfe 
(2011) 


tutors use it 
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The results from the focus group data further suggest that students perceive Turnitin as software 
that traditionally is aimed at “catching them out”, as opposed to a platform that can signpost 
developmental aspects of their learning; aligning software parameters with learning outcomes, and 
working to support the feedback process, would encourage new and possibly innovative ideas 
around impact and reach of online feedback. Software that can create a feedback audit, and safely 
store student feedback across modules and years of study, would greatly enrich the use of feedback 
for learning and serve to facilitate student engagement with the feedback process. Future research 
would do well to interrogate the features embedded into software such as Turnitin, and question 
how online feedback needs to develop to ensure that software works to enhance the learning process 
and not necessarily detract from it. 


